56 research outputs found
Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator
We consider the asymptotic behaviour of the marginal maximum likelihood
empirical Bayes posterior distribution in general setting. First we
characterize the set where the maximum marginal likelihood estimator is located
with high probability. Then we provide oracle type of upper and lower bounds
for the contraction rates of the empirical Bayes posterior. We also show that
the hierarchical Bayes posterior achieves the same contraction rate as the
maximum marginal likelihood empirical Bayes posterior. We demonstrate the
applicability of our general results for various models and prior distributions
by deriving upper and lower bounds for the contraction rates of the
corresponding empirical and hierarchical Bayes posterior distributions.Comment: 36 pages +24 pages supplementary materia
Fast Exact Bayesian Inference for Sparse Signals in the Normal Sequence Model
We consider exact algorithms for Bayesian inference with model selection
priors (including spike-and-slab priors) in the sparse normal sequence model.
Because the best existing exact algorithm becomes numerically unstable for
sample sizes over n=500, there has been much attention for alternative
approaches like approximate algorithms (Gibbs sampling, variational Bayes,
etc.), shrinkage priors (e.g. the Horseshoe prior and the Spike-and-Slab LASSO)
or empirical Bayesian methods. However, by introducing algorithmic ideas from
online sequential prediction, we show that exact calculations are feasible for
much larger sample sizes: for general model selection priors we reach n=25000,
and for certain spike-and-slab priors we can easily reach n=100000. We further
prove a de Finetti-like result for finite sample sizes that characterizes
exactly which model selection priors can be expressed as spike-and-slab priors.
The computational speed and numerical accuracy of the proposed methods are
demonstrated in experiments on simulated data, on a differential gene
expression data set, and to compare the effect of multiple hyper-parameter
settings in the beta-binomial prior. In our experimental evaluation we compute
guaranteed bounds on the numerical accuracy of all new algorithms, which shows
that the proposed methods are numerically reliable whereas an alternative based
on long division is not
A Bayesian nonparametric approach to log-concave density estimation
The estimation of a log-concave density on is a canonical
problem in the area of shape-constrained nonparametric inference. We present a
Bayesian nonparametric approach to this problem based on an exponentiated
Dirichlet process mixture prior and show that the posterior distribution
converges to the log-concave truth at the (near-) minimax rate in Hellinger
distance. Our proof proceeds by establishing a general contraction result based
on the log-concave maximum likelihood estimator that prevents the need for
further metric entropy calculations. We also present two computationally more
feasible approximations and a more practical empirical Bayes approach, which
are illustrated numerically via simulations.Comment: 39 pages, 17 figures. Simulation studies were significantly expanded
and one more theorem has been adde
Debiased Bayesian inference for average treatment effects
Bayesian approaches have become increasingly popular in causal inference
problems due to their conceptual simplicity, excellent performance and in-built
uncertainty quantification ('posterior credible sets'). We investigate Bayesian
inference for average treatment effects from observational data, which is a
challenging problem due to the missing counterfactuals and selection bias.
Working in the standard potential outcomes framework, we propose a data-driven
modification to an arbitrary (nonparametric) prior based on the propensity
score that corrects for the first-order posterior bias, thereby improving
performance. We illustrate our method for Gaussian process (GP) priors using
(semi-)synthetic data. Our experiments demonstrate significant improvement in
both estimation accuracy and uncertainty quantification compared to the
unmodified GP, rendering our approach highly competitive with the
state-of-the-art.Comment: NeurIPS 201
Debiased Bayesian inference for average treatment effects
No abstract availabl
Spike and slab empirical Bayes sparse credible sets
In the sparse normal means model, coverage of adaptive Bayesian posterior
credible sets associated to spike and slab prior distributions is considered.
The key sparsity hyperparameter is calibrated via marginal maximum likelihood
empirical Bayes. First, adaptive posterior contraction rates are derived with
respect to --type--distances for . Next, under a type of
so-called excessive-bias conditions, credible sets are constructed that have
coverage of the true parameter at prescribed confidence level and at
the same time are of optimal diameter. We also prove that the previous
conditions cannot be significantly weakened from the minimax perspective.Comment: 45 page
Stacked Penalized Logistic Regression for Selecting Views in Multi-View Learning
In biomedical research, many different types of patient data can be
collected, such as various types of omics data and medical imaging modalities.
Applying multi-view learning to these different sources of information can
increase the accuracy of medical classification models compared with
single-view procedures. However, collecting biomedical data can be expensive
and/or burdening for patients, so that it is important to reduce the amount of
required data collection. It is therefore necessary to develop multi-view
learning methods which can accurately identify those views that are most
important for prediction. In recent years, several biomedical studies have used
an approach known as multi-view stacking (MVS), where a model is trained on
each view separately and the resulting predictions are combined through
stacking. In these studies, MVS has been shown to increase classification
accuracy. However, the MVS framework can also be used for selecting a subset of
important views. To study the view selection potential of MVS, we develop a
special case called stacked penalized logistic regression (StaPLR). Compared
with existing view-selection methods, StaPLR can make use of faster
optimization algorithms and is easily parallelized. We show that nonnegativity
constraints on the parameters of the function which combines the views play an
important role in preventing unimportant views from entering the model. We
investigate the performance of StaPLR through simulations, and consider two
real data examples. We compare the performance of StaPLR with an existing view
selection method called the group lasso and observe that, in terms of view
selection, StaPLR is often more conservative and has a consistently lower false
positive rate.Comment: 26 pages, 9 figures. Accepted manuscrip
An asymptotic analysis of distributed nonparametric methods
We investigate and compare the fundamental performance of several distributed learning methods that have been proposed recently. We do this in the context of a distributed version of the classical signal-in-Gaussian-white-noise model, which serves as a benchmark model for studying performance in this setting. The results show how the design and tuning of a distributed method can have great impact on convergence rates and validity of uncertainty quantification. Moreover, we highlight the difficulty of designing nonparametric distributed procedures that automatically adapt to smoothness
- …